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DETAILED ACTION 

Response to Amendment 

1 . Applicant's response filed September 10 th 2008 has been entered and 
made of record. 

2. Applicant has not amended any claims. Claims 1-42 and 59-74 are 
cancelled. Claims 43-58 and 75-84 remain pending. 

3. Applicant's arguments in view of the presented rejection have been 
considered but are not found to be persuasive for at least the following reasons: 

Applicant argues that the combination of Cullen, Stubler and Shipp does not 
disclose the recited claims and that the combination of the references is improper. 
Examiner disagrees. What is essentially claimed in the recited independent claims is a 
library or database of images and associated texts. When new images are received, 
those images are compared to existing images and according to the comparison results 
the new image is grouped with the text associated with the best matching image. 
Cullen, Stubler and Shipp are all directed to the very well known practice of grouping 
and storing images and associated text and are therefore all essentially directed to the 
same field of endeavor. 

Applicant argues that Cullen discloses how to annotate images using text zones 
from a document, so therefore one of ordinary skill in the art would not be inclined to 
turn to Stubler to determine how to annotate images and therefore the combination of 
Cullen and Stubler is improper. Examiner disagrees. Cullen teaches text associated 
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with the images, which is essentially metadata. Stubler also teaches text associated 
with images as well as other metadata. What Stubler is relied upon to teach is 
associating the images and metadata to new images when similarity is determined. 
Stubler discloses a method of generating captions or text annotations for an image 
database as the images are entered into the database. Stubler discloses performing 
both low-level and high-level image processing to categorize images in order that the 
images can be compared with images already in the database (column 5, lines 19-36, 
column 6, lines 7-54 and column 7, lines 25-30). When the images are entered in the 
database for the first time they are compared with other images already in the database 
and when a match of a certain degree is determined, the newly entered images are 
annotated with the pre-existing text or captions associated with the images already in 
the database (column 8, lines 18-39). Stubler is cited to teach the claimed limitation of 
wherein upon receipt of a new image, the microprocessor compares the new 
image to the stored images (column 5, lines 19-36, column 6, lines 7-54 and column 
7, lines 25-30) and selects for inclusion in a 0 record the stored text associated 
with the stored image that is most similar to the new image (column 8, lines 18-39). 
Stubler discloses determining similarity of new images to already stored images and this 
is interpreted as the stored imaged that is most similar to the new image. 

It would have been obvious to on of ordinary skill in the art to enter the image 
documents and annotations into a database as taught by Stubler in combination with 
the database of Cullen in order that the image documents be entered and tagged in 
such a way that would group the images as taught by both Cullen and Stubler for better 
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categorization and searching of the database. Cullen and Stubler are combinable 
because they are both directed to storing images and associated text in an organized 
fashion. This is a well known motivation to one of ordinary skill in the art. 

Applicant further argues the use of the third reference to Shipp. Shipp is only 
cited to teach image and text storage in the field of medical records. It would be 
obvious to one of ordinary skill in the art to use the text and image association and 
storage of the references to Cullen and Stubler in the field of medical records. Shipp 
simply illustrates similar storage in the field of medical records. 

Applicant further argues that the Examiner has used improper hindsight 
reasoning in combining the cited references. Examiner respectfully disagrees. The 
motivation for storing images with associated text and for matching images to other 
similar images is not a novel concept. Anyone reasonably skilled in the art of image 
storage and classification understands the need and motivation to store images in such 
an organized fashion. The concept of linking images and associated text by comparing 
images to existing images is thoroughly illustrated in the cited references. No hindsight 
is needed to store images with like images and associated text. The rejection in view of 
the cited references is therefore maintained and remains FINAL. 

Double Patenting 

4. Applicant has filed Terminal Disclaimer in response to previously 
presented non-statutory obviousness-type double patenting rejection in view U.S. 
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Patent No. 6,735,329 to Shultz. The Double Patenting rejection is accordingly 
withdrawn. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 43-45, 47-51-58, 75-82 and 84 are rejected under 35 U.S.C. 
103(a) as being unpatentable over the combination of U.S. Patents 6,397,213 to Cullen 
et al. and 6,804,684 to Stubler et al. and 6,031 ,526 to Shipp 

With regard to claim 43, Cullen discloses a device for providing a text related 
to an image, comprising: 

A microprocessor (column 3, line 27); 

A library of stored images (database 222); and 

A library of stored texts (database 222), wherein each of the stored texts is 
associated with at least one of the stored images (column 3, lines 49- column 4, 
lines 1-65, Cullen discloses dividing the image document into zones representing the 
image portions and/or text portions. The text is associated with the image by attaching 
a caption or the header as discussed in column 4, lines 1-6). 
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Cullen discloses that the image division into zones and the annotation is typically 
performed as the documents are entered into the database or in other words upon 
receipt of a new image. Cullen does not go into detail about how the image portions are 
annotated when they are first entered into the database. Stubler discloses a method of 
generating captions or text annotations for an image database as the images are 
entered into the database. Stubler discloses performing both low-level and high-level 
image processing to categorize images in order that the images can be compared with 
images already in the database (column 5, lines 19-36, column 6, lines 7-54 and column 
7, lines 25-30). When the images are entered in the database for the first time they are 
compared with other images already in the database and when a match of a certain 
degree is determined, the newly entered images are annotated with the pre-existing text 
or captions associated with the images already in the database (column 8, lines 18-39). 
Therefore Stubler discloses the claimed feature of wherein upon receipt of a new 
image, the microprocessor compares the new image to the stored images (column 
5, lines 19-36, column 6, lines 7-54 and column 7, lines 25-30) and selects for 
inclusion in a 0 record the stored text associated with the stored image that is 
most similar to the new image (column 8, lines 18-39). It would have been obvious to 
on of ordinary skill in the art to enter the image documents and annotations into a 
database as taught by Stubler in combination with the database of Cullen in order that 
the image documents be entered and tagged in such a way that would group the 
images as taught by both Cullen and Stubler for better categorization and searching of 
the database. 
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Neither Stubler nor Cullen explicitly teaches that the database is that of medical 
records. Shipp teaches the use of combining images and text to create medical records 
to be stored (column 4, lines 1-10). It would have been obvious to one of ordinary skill 
in the art at the time of invention to use the combined database and database creation 
taught by Cullen and Stubler to store medical records in order to make the records more 
organized and searchable. 

With regard to claim 44, Cullen discloses wherein the new image, stored 
images and stored texts are digital (Figs. 1 and 2A, the images and texts are 
scanned into digital form). 

With regard to claim 45, Cullen discloses wherein the microprocessor uses 
digital image recognition to identify the one of the stored images that is most 
similar to the new image (col. 3, lines 24-39). Stubler also discloses digital image 
recognition for identifying similar images (column 7, lines 24-29 and column 5, lines 20- 
35 and column 6, lines 6-54). 

With regard to claim 47, the discussion of claim 43 applies. Shipp discloses that 
the medical record is concerned with recording images of a procedure (column 2, lines 
29-37). 
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With regard to claim 48, Shipp discloses that the procedures are performed by a 
surgeon and are thus judged to be surgical procedures (column 4, lines 1-10). 

With regard to claim 49, the discussion of claim 45 applies. 

With regard to claim 50, Shipp discloses that the images are real-time (column 3, 
lines 12-20). 

With regard to claim 51 , the discussions of claim 43, 45 and 47 apply. 

With regard to claim 52, Cullen, Stubler and Shipp all disclose that the stored text 
is descriptive of the new image. See Cullen (column 4, lines 30-39), Stubler (column 5, 
lines 20-36), and Shipp (column 4, lines 1-10). 

With regard to claim 53, the discussion of claim 44 applies. 

With regard to claim 54, Cullen discloses means for communication the images 
and texts (network interface 144, column 3, line 33). 

With regard to claim 55, Shipp discloses wherein the images depict aspects of 
medical procedures (column 3, lines 13-27). 
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With regard to claim 56, the discussion regarding claim 43 applies. Cullen and 
Stubler disclose matching of images and text in databases and Shipp teaches 
combining video images and text descriptions into a digital image record of a surgical 
procedure. These kinds of records would be compatibly searched and matched by the 
combination taught by Cullen and Stubler. 

With regard to claim 57, and 58, Cullen and Stubler and Shipp disclose capturing 
and processing multiple images, processing those images and using them to create a 
database. 

With regard to claim 75, the discussion of claim 43 applies. 
With regard to claims 76-80, the discussions of claims 43-46 apply. 
With regard to claim 81 , the discussion of claim 43 applies. 
With regard to claims 82 the discussion of claim 43 applies. 
With regard to claim 84, the discussion of claim 43 applies. 
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6. Claims 46 and 83 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of U.S. Patents 6,397,213 to Cullen et al., 
6,804,684 to Stubler et al., 6,031 ,526 to Shipp 6,539,61 7 to Prokoski 

With regard to claim 46, the combination of Cullen, Stubler and Shipp disclose 
the method of claim 43, but do not disclose the wherein, when no stored image is 
similar to the new image, the microprocessor generates a signal to indicate that 
no match has been identified. When attempting to match an image, it very well 
known in the art to indicate no match when the matching process fails. Prokoski 
discloses matching medical images and teaches the use of a failure to match signal 
(column 16, lines 41-45). Therefore it would have been obvious to one of ordinary skill 
in the art at the time of invention to use the signal indicating no match has been found 
as taught by Prokoski in combination with the image matching processing taught by 
Cullen and Stubler for use with the medical images of Shipp in order to determine when 
no match was achieved. 

Contact Information 

7. Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to WESLEY TUCKER whose telephone number is 
(571)272-7427. The examiner can normally be reached on 9AM-5PM. 
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organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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